1. Field of the Invention
The present invention generally relates to image processing apparatuses and image processing methods, and particularly relates to an image processing apparatus and image processing method for enlarging and reducing an image.
2. Description of the Related Art
Enlargement and reduction of a digital image is a process of resampling that changes the sampling intervals corresponding to pixel positions of an original image into different sampling intervals. In general, such process requires that the pixel values of the original image be interpolated by newly generated pixel values (i.e., new pixel values are generated from the pixel values of the original image to be inserted between the pixels of the original image). Examples of such interpolation include nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, etc.
Nearest neighbor interpolation uses the value of a pixel closest to a pixel of interest for which a pixel value needs to be obtained, as the pixel value of this pixel of interest, and is the simplest interpolation method. Bilinear interpolation obtains the pixel value of a pixel of interest by applying linear interpolation to the values of four neighboring pixels situated at the top, bottom, left, and right of the pixel of interest for which a pixel value needs to be obtained. Bicubic interpolation uses a cubic function for interpolation. In general, a cubic polynomial is calculated based on the pixel values of 16 neighboring pixels (four pixel by four pixel) surrounding a pixel of interest for which a pixel value needs to be obtained, thereby obtaining the pixel value of this pixel of interest. There are other interpolation methods such as a method that uses a sinc function. From the viewpoint of the balance between the amount of computation and the quality of an image, bicubic interpolation is typically used as a preferred method that produces satisfactory results.
Digital cameras are provided with digital zoom functions. In digital zoom processing, a portion of image data captured by the CCD/CMOS image sensor is cut out, and this extracted partial image is enlarged or reduced so as to become an image of desired image size. In conventional configurations, a color processor performs an image cutout process on the output data of the image sensor, and the image data is then temporarily stored in a frame memory, followed by using a scaler provided as a separate module to perform enlargement or reduction.
The color processor is a circuit that receives image data from the image sensor and performs signal processing such as defect pixel correction for correcting defects by processing the data of defect pixels contained in image signals, color interpolation for obtaining color data with respect to each pixel based on color information from the RGB Bayer array, shading correction for correcting lens distortion based on the color data, etc. The scaler is provided as a module separate from the color processor. The scaler serves to perform image enlargement or reduction by applying bicubic interpolation or the like to the image data stored in the frame memory.
In the conventional configuration as described above, the frame memory is provided as an intervening element between the cutout process performed by the color processor and the enlargement or reduction process performed by the scaler. This is because it is difficult to perform enlargement at arbitrary magnification in a sequential manner by matching the speed of the output of the image sensor. The output of the image sensor is supplied at predetermined speed as scan line data that has been scanned in the main scan direction. If the image data is to be processed for enlargement or reduction without being stored in a frame memory, such processing needs to be performed at the same speed as the speed at which the image data stream is supplied as scan line data.
In order to process line data output from the image sensor sequentially by matching the data supply speed, processing on each pixel or each image block needs to be performed repeatedly at constant intervals. In image enlargement at arbitrary magnification, however, the pixel value of the same pixel needs to be referred to more than once for interpolation to increase the number of samplings. Also, in the case of enlargement by a factor that is not equal to an integer, the number of times that the same pixel value is referred to may differ from pixel to pixel.
When enlargement by a factor of 1.5 is performed by use of nearest neighbor interpolation, for example, the (n−1)-th pixel is referred to once, the n-th pixel referred to twice, and the (n+1)-th pixel referred to once, which is repeated for horizontal enlargement. Further, each pixel of the (n−1)-th line is referred to once, each pixel of the n-th line referred to twice, and each pixel of the (n+1)-th line referred to once, which is repeated for vertical enlargement. In this manner, the number of times that a pixel value is referred to is not constant, and, also, such number varies in response to a magnification factor. This makes it difficult to perform sequential processing at constant intervals with respect to any given magnification factor.
In consideration of this, the conventional digital zoom processing uses an intervening frame memory between the image cutout process and the enlargement or reduction process, thereby making it possible to perform enlargement/reduction without requiring a needlessly complex algorithm and circuit configuration. Such configuration, however, have a memory access occurring between the image cutout process and the enlargement/reduction process, resulting in inefficient processing. Further, since intermediary data is stored in memory, an excess memory area needs to be set aside.
Patent Document 1 discloses a technology that improves the speed of a cutout process by eliminating the need to make settings to the horizontal and vertical pixel sizes and the need to convert the number of pixels. Patent Document 2 discloses a technology that is used for the configuration in which sensitivity is improved by pixel summation when a picture of a poorly illuminated object is taken. This technology produces high sensitivity image data having desired size by resizing image data by n times horizontally and m times vertically, compared to the image size that is to be produced and displayed after pixel summation, followed by performing n-horizontal-pixel-by-m-vertical-pixel pixel summation after the resizing.
[Patent Document 1] Japanese Patent Application Publication No. 2004-282305
[Patent Document 2] Japanese Patent Application Publication No. 2005-115598
Accordingly, there is a need for an image processing apparatus that can perform image enlargement or reduction at any desired magnification by processing line data sequentially by matching its data supply speed as the line data is scanned on an image in the main scan direction.